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(57) Abstract: A hand held electronic device having at least two displays 
is disclosed. At least one display is a direct-view display, like those on most 
cell phones or PDAs in use in 2001, for viewing at normal reading distance 
of approximately 12 to 24 inches ("arms' lenght" viewing). At least one of 
the other displays is a microdisplay. a tiny display with magnifying optical 
elements, for viewing larger, higher-resolution images when the microdis- 
play is positioned close to the eye. The invention allows coordinating mi - 
crodisplays and direct-view displays in ways that allow people to comfort- 
ably access and interact with full-page Web content on pocket-size devices. 
When a user is viewing a Web page (or other content) on a device's mi- 
crodisplay held near-to-eye, the device allows the user to position a cursor 
or a rectangular outline (or some other indication of a "region of interest") on 
a particular part of the Web page, and then when the user moves the device 
out to arms' -lenght viewing, the user should be able to view that region of 
interest on the direct-view display- that is, view a subset of the larger image 
that appeared on the microdisplay. 
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COORDINATING IMAGES DISPLAYED ON DEVICES WITH TWO OR MORE DISPLAYS 
RELATED APPLICATIONS 

This application claims priority, under 35 USC § 120, from US Provisional Patent Application No. 
60/280,791 , filed on April 2. 2001 . 

5 BACKGROUND OF THE INVENTION : 

The invention generally relates to hand-held communication and computing devices, such as cellular- 
phone handsets, smart-phones, Web-phones, and PDAs. More specifically, the invention relates to 
systems and methods for coordinating images displayed on hand-held devices that have two or more 
displays, especially those that include a direct-view display for viewing relatively low-resolution text or 
10 images (like the displays built into most cellular phones today), and a microdisplay that uses 

magnifying optics to display images that, when the microdisplay is held near a user's, eye can appear 
larger and higher resolution than the images shown on the direct-view display. 

Using recently developed high-resolution microdisplay module technology, device makers can embed 
microdisplays in devices, or attach microdisplays including wearable displays to devices, and allow 
15 users to view full-size web-pages on pocket-size devices - as well as being able to view large video 
and still images, blueprints, office documents, and other high-resolution content. 

Device makers have experimented with embedding microdisplays in devices and attaching 
microdisplays to devices (including attaching wearable near-to-eye displays). However, prior to this 
invention, it has been awkward and difficult to navigate and interact with full-size Web pages and 

20 other documents and applications displayed on the microdisplays of microdisplay-enhanced hand- 
held devices. Devices with embedded microdisplays must be held near-to-eye to see the content on 
the microdisplays. While users do not mind briefly bringing a device near-to-eye to view content (after 
all, that is what we do with cameras), users do not want to hold a device near to eye for longer periods 
of time. If a user is spending five minutes, ten minutes, or more reading or interacting with a set of 

25 Web pages or other documents, and if they have to hold a device near-to-eye continuously during that 
time, they will grow uncomfortable with the device. Furthermore, typing text on a hand-held device 
(already a relative awkward activity) is exceedingly awkward when the device has to be held near-to- 
eye to see the text as you type it. Devices with attached wearable displays, such as the head- 
mounted monocular displays that Xybemaut Corp. uses in their wearable computer systems, allow 

30 users to comfortably keep microdisplays near-to-eye for longer periods of time (relative to hand-held 
devices with embedded displays); but users still find it awkward to type text into documents viewed on 
wearable displays. Typing text into a Web page or other document displayed by a device on a 
microdisplay typically involves either using a cursor control to "type" on a virtual keyboard image 
shown on the microdisplay (e.g. moving the cursor over each character displayed on the virtual 
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keyboard and clicking), or using a separate physical device (such as a keypad, a keyboard, a tablet or 
writing pad, or a virtual keyboard). Typing using a virtual keyboard displayed on the microdisplay is 
tedious and slow (particularly when holding a device near-to-eye) because it requires the user to 
carefully position the cursor over each little character on the virtual keyboard to type each character. 
5 Further, using a mouse on a regular computer to type text on a virtual keyboard is awkward, and it is 
even more awkward when using a near-to-eye microdisplay. Even when a user can type text on a 
physical keypad while looking at a document on a wearable display, if user's haven't learned to touch- 
type well (i.e. type without looking at the keypad), then users can experience an uncomfortable mental 
disconnect as they frequently shift their eye (or eyes) between the keypad and the image displayed on 
1 0 the wearable display's near-to-eye microdisplays. 

However, the potential benefits of using near-to-eye microdisplays with hand-held devices are 
tremendous: They can allow users to view full-size web pages and other high-resolution content on 
pocket-size devices. So solving the problems outlined above would enable the development of 
compelling new types of hand-held devices. This patent describes inventions that help solve these 
15 problems. 

A device with which the invention can be implemented is disclosed and claimed in U.S. patent 

application S/N . filed on even date herewith, assigned to the common assignee, 

and incorporated herein by reference. 

In devices using the present invention, at least one display is a direct-view display, like those on most 

20 cell phones or PDAs in use in 2001 , for viewing at normal reading distance of approximately 12 to 24 
inches (which this patent will refer to as "arms'-length" viewing). And at least one of the displays is a 
microdisplay, a tiny display with magnifying optical elements, for viewing larger, higher-resolution 
images when the microdisplay is positioned close to the eye. The microdisplays can be embedded in 
devices, or they can be attached to the devices, or the microdisplays can be in wearable displays 

25 attached to the devices (wirelessly or through a cable). Many digital video cameras use microdisplays 
for their view-finders: When a user brings the view-finder to their eye when the camera is playing back 
video, the user can watch a larger image of the video-clip than could be seen on small direct-view 
display. In recent years, high resolution microdisplays have been developed that can present images 
large enough to display most Web pages. For example. Inviso Inc.'s 3430 Optiscape II microdisplay 

30 displays an 800x600 pixel color image that appears to users to be as large as a 19-inch monitor 

roughly 2.5 feet away. Since optics are used to create an image that looks large, despite the fact that 
the microdisplay module is small, these images are often called "virtual images". And in this patent we 
will sometimes call them "apparent images". Preferred embodiments of this invention would normally 
be implemented on devices using microdisplays that can display color images at least 800 pixels wide 

35 and at least 600 pixels tall. 
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An important common characteristic of the preferred embodiments of the present invention is that they 
are designed to facilitate a seamless, natural transition between near-to-eye viewing of content on a 
device's microdisplay (or microdisplays) and arms'-length viewing of content on a direct-view display. 
Key to enabling this seamless transition is software, hardware and storage mechanisms that are used 
to intelligently coordinate images on the microdisplay with images on the direct-view display. Using 
the descriptions in this patent and the figures for this patent (which help illustrate some of the 
graphical user interface elements of the invention disclosed herein), ordinary hardware and firmware 
engineers skilled in the art of developing computing and communication devices will be able to 
implement the hardware, and storage inventions disclosed herein, and ordinary programmers skilled 
in the art of software engineering on hand-held devices will be able to write their own code to 
implement the software inventions disclosed herein for devices that embody the hardware and 
storage inventions disclosed herein. 



A preferred embodiment of my invention is a "smart-phone" supporting functions such as instant 
messaging, email, and contact management, in addition to phone calling, with both a microdisplay 

1 5 and a direct-view display. Since most instant messages, for example, are relative short text 

messages, this smart-phone could display instant messages on the direct-view display so that users 
can see the messages at ami's length, and can concurrently see the keys, buttons and controls on 
the phone handset so the user can easily type replies to the messages. When an instant message 
includes a reference to a Web page URL, or includes an attached photo or other relatively large high- 

20 resolution image or document, then when the user chooses to view that Web page or attachment, the 
smart-phone can display the Web page or attachment in the microdisplay instead of the direct-view 
display. This allows the user to see the Web page or attachment, which may be relatively large and 
high-resolution, by bringing the microdisplay to the user's eye, much like a PC display like image. 

Additionally, when the device is displaying an image (or is preparing to display an image) in one 
25 display but not the others, then the device can display a text or graphics indication on one or more of 
the displays that is not displaying that image, indicating that one of the other displays is displaying that 
image (or that the device is preparing to display that image). For example, if the user sees an instant 
message in the direct-view display (on a device with one direct-view display and one microdisplay), 
and that instant message refers to a Web page URL, if the user chooses to view that Web page, then 
30 while the device is downloading or preparing the Web page for display on the microdisplay the device 
can display a flashing eye icon on the direct-view display; and when the Web page is ready for 
viewing on the microdisplay, then the device can display a static icon on the direct-view display. This 
lets the user know that the user can now bring the device near-to-eye to view the Web page in a high 
resolution, magnified form. The same technique can be used any time the user can operate functions 
35 using the direct-view display that result in content then appearing on the microdisplay. or vice-versa. 
(If the user has the device near-to-eye and operates a function that results in new content appearing 
on the direct-view display, the microdisplay can display some indication that there is new content to 
view on the direct-view display.) It is important to provide these cues, since a user typically can not 
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see the content on a microdisplay when holding the device at arms-length, and a user typically can 
not see content on a direct-view display (or doesn't think to look at the direct-view display) when 
looking into a microdisplay that is near-to-eye. 

Device designers can choose to use the direct-view display as the primary display for many smart- 
5 phone functions, including instant messaging, email, phone dialing, contact management, schedule 
management, calculator functions, and the like, and to use the microdisplay as a high resolution 
viewer coordinated with the direct-view display. 

In the emerging age of Internet-enabled mobile devices, some of the most important embodiments of 
this invention involve coordinating microdisplays and direct-view displays in ways that allow people to 

10 comfortably access and interact with full-page Web content on pocket-size devices. One important 
way our invention does this is as follows: When a user is viewing a Web page (or other content) on a 
device's microdisplay held near-to-eye, the device should allow the user to position a cursor or a 
rectangular outline (or some other indication of a "region of interesr) on a particular part of the Web 
page, and then when the user moves the device out to arms -length the user should be able to view 

15 that region of interest on the direct-view display - that is, view a subset of the larger image that 

appeared on the microdisplay. This patent will refer to this subset as a "region of interest subset" or a 
"region of interesr. 

When the direct-view display displays this region of interest, the direct-view display acts as a window 
onto the larger image that the user had been viewing on the microdisplay. That is, the region of 

20 interest portion of the image seen on the microdisplay is mirrored on the direct-view display. The user 
should then be able to read text or interact with content in that region of interest, or use controls on 
the device to scroll the region of interest left, right up, down, or other directions, to see different parts 
of the image other than the subset of the image currently shown in the direct-view display. If the user 
moves the region of interest while viewing the image in the direct-view display, then when the user 

25 returns to viewing the image in the microdisplay, the device can graphically indicate where the region 
of interest has moved - for example, by overlaying a rectangle in the shape of the region of interest 
(which will generally correspond to the pixel width and height of the direct-view display) on top of the 
image seen in the microdisplay. Device makers can choose not to explicitly show a rectangle or other 
explicit representation of the region of interest on the image viewed in the microdisplay, or choose to 

30 make this kind of explicit "region of interesr indicator optional for the user. Whether or not the region 
of interest is explicitly outlined on the image viewed in the microdisplay, the device can mirror the 
region of interest in the direct-view display, as described above. Also note that the region of interest 
does not necessarily have to be rectangular. For example, a device designer could choose to use a 
circular or other shaped direct-view display (for example, on a watch or a non-conventional cell-phone 

35 or PDA), and then choose to use a correspondingly shaped region of interest. 
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The usage model facilitated by this invention mirrors what happens when a person uses a desktop 
computer monitor: Typically a person briefly "takes in" the entire monitor display (i.e., looks at the 
entire image without focusing attention on any specific region of the display at first), and then focuses 
attention on a specific region of interest (such as text they want to read, or a text-insertion point where 
5 they want to type, or some other item of interest). This invention allows microdisplay-enhanced 
devices to model this behavior. 

As a result, this invention enables better ergonomics and better power-consumption characteristics 
than previous designs for many types of microdisplay-enhanced devices. With this invention, a user 
can choose to briefly observe a large area of content displayed on the microdisplay, and quickly skim 

10 the large area of content to identify a region of interest where they want to focus attention - and then 
(if they want) they can bring the device to arms'-length to view and interact with that region of interest 
on the direct-view display. By allowing the user to view a large area of content on the microdisplay, 
users can quickly orient themselves and quickly skim the large area of content to find a specific region 
of interest - just as they do on desktop monitors. This is something users cannot do on the small 

15 direct-view displays available on pocket-sized phones. PDAs, and hand-held computers that do not 
include embedded or attached microdisplays. Then, by allowing users to move the region of interest 
on the direct-view display held at arms-length, users can choose to do most of their reading, typing, 
and other interacting with content while holding the device at arms'-length (i.e. normal reading 
distance). Being able to frequently operate the device and interact with Web pages and other content 

20 while holding the device at arms -length is more comfortable for many people than having to hold a 

device near-to-eye for long periods. And it is much easier to type text on a device held at arms-length 
than a device held near-to-eye. In addition, when the user is not holding the device near-to-eye, the 
microdisplay can be turned off or idled, saving power - so devices that use this invention can use 
significantly less power than devices that leave microdisplays on continuously. Similarly, when the 

25 user is holding the device near-to-eye, direct-view displays on the device can be turned off or dimmed 
or idled, saving power. 

Clearly this method of intelligently coordinating content on microdisplays with content on direct-view 
displays can be applied to a wide range of content in addition to Web pages - including content such 
as spreadsheets, presentations, word processor files, blueprints, video, database forms, and other 
30 types of images, documents and applications. 



The primary motivitation for this invention has to make it feasible for people to comfortably access and 
interact with full size Web content (and other high-resolution or large visual content) on pocket-size 
devices with built-in or attachable microdisplays. But the present invention's embodiments related to 
intelligently coordinating direct-view displays and microdisplays are applicable even when the 
35 microdisplays are part of a wearable display connected to the device rather than being embedded in 
or attached directly to the device. Wearable displays can include one or two microdisplays. For 
example, Inviso Inc.'s binocular wearable display product concept called "eShades" included two 
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microdisplays, one for each eye. If a user is wearing a wearable display and looking at the near-to-eye 
microdisplays of the wearable display, they may find it awkward to type, particularly if the user has to 
look at a device's keypad frequently to find the letter keys while typing. A monocular wearable display 
(i.e. one with just one microdisplay positioned in front of one eye) lets the user use the other eye to 
look at the keypad; and many binocular wearable displays (like Inviso Inc.'s eShades concept 
product) allow users to see below the displays and therefore see a device's keypad held below the 
field of view of the wearable displays. But it is awkward to have to continuously shift ones eyes 
between the keypad and the image presented by the microdisplay while typing. Most users would be 
more comfortable if the region surrounding the current text-entry point (which might be an editable text 
field on a Web-based form, for example) were mirrored in the direct-view display of the device the 
user is using to type, ideally with the direct-view display close to the keypad on the device. This allows 
the user to keep their eyes focused in a small area on the device while typing. 

BRIEF DESCRIPTION OF THE DRAWINGS : 

Fig. 1 A is a side view of a microdisplay-enhanced handset. 

Fig. 1B is a front view of a microdisplay-enhanced handset 

Fig. 1C is a front view of a microdisplay-enhanced handset, held in a person's hand. 

Fig. 2A is a side view of a flip-style handset with both a direct-view display and a microdisplay. 

Fig. 2B is a front view of a flip-style handset with both a direct- view display and a microdisplay. 

Fig. 2C is a front view of a flip-style handset with both a direct-view display and a microdisplay, held in 
a person's hand. 

Fig. 3A is a side view of an alternative configuration for a handset with both a microdisplay and a 
direct-view display. 

Fig. 3B is a front view of an alternative configuration for a handset with both a microdisplay and a 
direct-view display. 

Fig. 3C is a front view of an alternative configuration for a handset with both a microdisplay and a 
direct-view display, held in a person's hand. 

Fig. 4 is an illustration of a handset with two displays and with buttons for highlighting and selecting 
selectable items on a list of selectable items displayed on one of the displays. 
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Fig. 5 is an illustration of a handset according to my invention, showing the coordination of images on 
the direct-view display and on the microdisplay. 

Fig. 6 is an illustration of a handset according to my invention, showing an indication on the direct- 
view display (shown in close up on the left of Fig. 6) that there is new content ready for viewing on the 
5 handset's microdisplay. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS : 

The following describes a handset on which my invention can be embodied. Fig. 1A shows a 
microdisplay-enhanced handset 1 18 in side-view. Fig. 1B shows the same in front-view. Fig. 1C 
shows the same in front-view, shown held in a person's hand. This handset includes a microdisplay 
10 100 positioned above a direct-view display 101. Most of the elements of this handset could be 

applied to a wide range of devices - including cellular phones, "Smart-phones". Web-phones, PDAs, 
hand-held computers, remote controls, and others. Throughout the rest of this patent, most 
references to "handsets" can refer to any of these types of devices. 

The direct-view display can be any kind of direct-view display, but preferably it would be a thin bit-map 
1 5 LCD display with less resolution than the microdisplay but enough to render an identifiable portion of a 
larger image. For example, a 160x1 20-pixel direct-view display with 3-bits of gray scale (white, black, 
and six intensities of gray in-between white and black) would be sufficient for displaying text and 
simple graphics while still keeping the cost of the direct-view display relatively low. 

Below the side-view of Fig. 1A and the first front-view illustrations of the handset in Fig. 1B are side- 
20 view 116 and front-view 117 illustrations of Inviso's 3430 Optiscape microdisplay module which 

includes electronics and optics - just to illustrate their size relative to the overall handset. Many other 
elements on this handset are common on cell phone handsets today such as the speaker 1 14, a 
power on/off button 108, a microphone 115, "Start Call" and. "End Call" buttons 112. 

Handset users frequently interact with various functions supported on the handset, such as instant 
25 messaging, email, contact management, dialing from a list of contacts, calculator, Web browsing, and 
so on. To interact with those functions, users need to see (or hear) feedback as they use the 
functions. For example, when typing an instant message, a user needs to see the text the user is 
typing to know that it *s being typed correctly. For most functions, including instant messaging, the 
feedback presented by the handset is relatively simple and brief - capable of being expressed in a few 
30 lines of text or in a small low-resolution bit-mapped display. On handsets built according to this 

invention, relatively simpler or brief feedback should usually be presented on the direct-view display 
100 of Fig. 1B, as is done on a traditional handset that has only a direct-view display. This allows 
users to see the feedback while holding the handset at arms* length (as opposed to near-to-eye) so 
they can still easily see and operate the handset's buttons and other controls. But when a function 
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involves viewing or interacting with a large image, such as a Web page, a spreadsheet, or a high- 
resolution document attached to an email, then a handset built according to this invention should 
display that image on the microdisplay 101 of Fig. 1B. 

While different handsets can differ in what they display on the two displays (the microdisplay 101 and 
5 the direct-view display 100), in general designers should strive to allow users to interact with the 
handset's functions while viewing the direct-view display. That makes it easier to see and operate 
more of the device's buttons and controls (e. g. for typing or for navigating through the functions* 
features) as compared to holding the device near-to-eye. Of course, designers can also allow users 
to interact with functions while holding the device near-to-eye and using the microdisplay by designing 
1 0 the handset so it displays some feedback on the microdisplay (instead of. or in addition to, displaying 
that feedback on the direct-view display). 

With a one-piece handset such as the one illustrated in Fig. 1 , there are two advantages to placing the 
microdisplay near the top of the handset and placing the direct-view display in the middle of the phone 
(above the keypad). First, it makes it easier for the user to bring the microdisplay close to the eye. as 
1 5 compared to placing the microdisplay in the middle or at the bottom of the handset Second, it places 
the direct-view display (where feedback for most text being typed would be presented) near the 
keyboard (so the user does not have to move their eyes far from the typing keys to the display 
showing the feedback). 

The device can also be designed such that depressing a given button on the device (or depressing a 
20 combination of one or more buttons or controls) results in the device popping up and displaying (on 
one or more displays) a menu in a menu bar associated with the device or with the front window on 
the device (such as the "File" menu at the top left of documents on most PC documents) while 
concurrently highlighting the top menu item in that menu. Then depressing left or right arrow buttons 
on the device, such as those in control 1 13 on Fig. 1, could closes the currently popped-up menu and 
25 causes popping up the previous or next menu in the menu bar and highlights the top item on the 

menu. Further, the device could be designed such that further depressing up or down arrows on the 
device (such as those in control 1 1 3 on Fig. 1 ) highlights the next or previous menu item in the 
currently popped up menu. At that point, pressing an "enter" or "select" button on the phone results in 
the highlighted menu item being selected. Different types of items can do different things when 

m 

30 selected in this way. Typically, selecting an item should cause the same behavior as would occur on a 
PC if the user used a mouse to click on that type of item. For example, selecting a check box that is 
not currently checked should cause it to be checked; and selecting a check box that is currently 
checked should cause it to be unchecked. 

The device can further be designed such that spoken commands via speech recognition software can 
35 be used to pop up and traverse menus or to select menu items; and in which the device's speech 
recognition features are enabled only when the user holds in a button designed for that function on 
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the device. This "push-to-talk" button could be on the side of the device where a user could depress it 
with a finger or thumb on the hand holding the device. 

Fig. 2A through Fig 2C illustrate a flip-style handset with both a direct-view display 200 and a 
microdisplay 201 . Microdisplays are relatively thick (because of their magnifying optical elements) 
5 and direct-view displays are relatively thin (since they are lower resolution and do not require 

magnifying optical elements). So a flip-style phone would place the direct-view display on the thin 
part of the phone that flips away from the main body of the phone. In Fig. 2B, this is the top part of 
the phone. A flip-style phone could also be designed in which the keyboard and the direct-view 
display are on a relatively thin bottom part of the phone, and the microdisplay is on a relatively thick 
10 top part of the phone that flips up. But that would tend to make the phone feel top-heavy. 

Fig. 3A through Fig. 3C illustrate a compelling alternative configuration for a handset with both a 
microdisplay 300 and a direct-view display 301 as seen in the side-view representation of Fig. 3A. In 
this embodiment, the direct-view display is placed directly over the microdisplay. As noted above, for 
most functions (placing a phone call, scrolling through lists of contacts, and instant messaging, for 

1 5 example) the user simply uses the direct-view display for feedback, since the feedback consists of 
relatively low-resolution text or images. In these instances, when the user looks at the front of the 
phone 305 In Fig. 3B, he or she see the relatively low-resolution text or images on the direct-view 
display 302. But when the user decides to view an image or document that is best viewed on full- 
screen, high resolution displays (Web pages, excel spread-sheets, large photos, etc.) then the 

20 handset would display that image or document on the microdisplay. A handset can be designed to 

have a button (or other control) that the user uses to make the direct- view display become transparent 
(and to turn the microdisplay on) so the user can see through to the microdisplay. But a preferred 
embodiment would include a distance-sensor 304 in Fig. 3B that automatically senses when the 
user's eye is close to the microdisplay (within 2-3 inches) and then automatically makes the direct- 

25 view display (302 in Fig. 3A and Fig. 3B) transparent and turns on the microdisplay (300 in Fig. 3A 

and 303 in Fig. 3C) so the user can see through the direct-view display 301 to the image presented by 
the microdisplay 300. Various distance-sensing technologies - including ultrasonic distance sensors - 
are used in current digital video cameras, still cameras, and other electronic devices. For example, 
the Sony Mavica MVC CD1000 digital still camera has both a direct-view display on its back and a 

30 separate mid-resolution microdisplay in its viewfinder, and it includes a distance sensor so that when 
a user puts their eye close to the viewfinder the microdisplay-based viewfinder automatically turns on. 
This handset of Fig. 3A through Fig. 3C would automatically turn on the microdisplay only when the 
handset is displaying a high-resolution image and when the sensor senses that the user's eye is close 
enough to the microdisplay - within three inches or so. At all other times, the handset would 

35 automatically turn the microdisplay off. saving considerable power relative to a device that leaves the 
microdisplay on ail the time or one that leaves it on even when the user's eye is not close to the 
microdisplay. As used herein, the phrase "turn the microdisplay off 1 means to deactivate the 
microdisplay such that it is not generating images or generating any light that would prevent the 
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correct operation of the direct-view display. It does not necessarily mean that no power is being 
supplied to the microdisplay. However, the result is substantial power savings. 

This handset of Fig. 3 A through Fig. 3C can also be built less tall than handsets that separate the two 
displays (as in Fig. 1 A through Fig. 1C), since the two displays overlap one another in this design, 
5 although it could make the phones a bit thicker when viewed from the side. 

A handset could also be designed with a direct-view display on one side of the handset (such as the 
front) and a microdisplay on the other side of the handset (such as the back). But this would require 
users to turn the handset over when switching between viewing the direct-view display and viewing 
the microdisplay - which is more awkward than simply bringing the handset to the eye to view an 
10 image in the microdisplay. 

On any of these dual-display devices, either or both of the displays could be touch sensitive (like 
many PDAs today), allowing users to input data and control aspects of the device by moving their 
fingers, a pen, or a stylus across the touch-sensitive area of the display(s). 

Each of the handsets illustrated in Fig. 1A through Fig. 3C is shown with three side-buttons on the 
1 5 right side -"Shift", "2nd Letter", and "3rd Letter" side-buttons - which a user can press with the hand 
holding the phone while simultaneously using the other hand to type the handset's face-keys. For 
example, to type the lowercase letter "m", the user would just type the "6" key. To type the uppercase 
letter "M", the user would type the "6" key while holding in the "Shift" side-button. To type the lower- 
case letter "u", the user would type the "8" key while holding in the "2nd Letter" side-button. To type 
20 the upper-case letter "U", the user would type the "8" key while holding in both the "Shift" side-button 
and the "2nd Letter" side-button. This can allow for faster text-typing than on current cell phones and 
PDAs. This is important for typing -intensive applications and functions such as instant messaging, 
email, interacting with some Web pages, and other applications that involve entering information into 
forms or documents. That fast-typing-enabling invention is the subject of patent application S/N 

25 , filed , assigned to the common assignee and incorporated herein by 

reference. 

The device can also include a roller control 102 of Fig. 1 B which, when rolled in one direction, 
changes the highlighting in a displayed list of selectable items from the currently highlighted item to 
the next selectable item; and, when rolled in the other direction, changes the highlighting from the 

30 currently highlighted item to the previous selectable item. This is similar to using the Tab-forward and 
Tab-back keys on a Windows PC to move the highlighted item on a Web-page, dialog or other 
document - often called the "focus" by programmers - from one selectable item to the next one or to 
the previous one. Note the "list" of selectable items does not have to appear linear A web-page has 
a "list" of selectable items, as seen in the Web-page's HTML, even if those items are arranged 

35 nonlinearly in a two-dimensional design. In fact, a "selectable" item can be any displayed item that 
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can either be selected or, in the case of a text-entry box, typed into. The device can also be designed 
to apply to items that are part of Web pages and other documents, as well as to items that are 
displayed by the handset itself such as the list of functions available on the phone device. In alt these 
cases, a roller control can also be used to traverse through lists of items and to select items. The 
5 roller control 102 can also be pressed to select the highlighted item in any displayed list of items. 



As seen in Fig. 4, the device includes a button 401 that, when pressed briefly, changes the 
highlighting in a displayed list of selectable items from one selectable item to the previous one. This 
patent also refers to the highlighted selectable item as the "in-focus" item, the "item in focus", or the 
"focus" item. The device can also include a button 403 that, when pressed briefly, changes the 
highlighted item in a displayed list of selectable items from one selectable item to the next one. The 
device can also include a button 402 that, when pressed, selects the highlighted item in the displayed 
list of items. These buttons do not have to be configured as buttons 401 , 402, and 403 appearing in 
Fig. 4: They could be configured as arrow keys and other buttons elsewhere on the device - such as 
the arrows and enter buttons 1 13 on Fig. 1B. Each of these buttons can be multiple-speed (which this 
patent will also refer to as "multi-speed") - for example, moving the in-focus item quickly from the 
currently highlighted selectable item to the next or previous one when pressed hard, but moving the 
highlighting slowly when pressed lightly. These buttons can go by many names - such as Tab-forward 
and Tab-back, up and down, and so on. This patent will refer to them genetically as up and down or 
up/down. If they are multiple-speed, this patent will refer to them as multi-speed up/down controls. 
The buttons can be placed under a shared length of plastic to form a toggle switch, with one button 
under one side of the toggle and the other button under the other side of the toggle, as shown in 
Fig. 5. 



As additional examples of operation of the above buttons, when the button 401 (or 403) of Fig. 4 is 
pressed and held for more than a defined period of time (for example, more than 0.5 second), then 
instead of the highlighting moving from one selectable item to the previous (or next) item just once, 
the highlighting moves to the previous (or next) selectable item, stays briefly (e. g. 0.25 sec), moves to 
the previous-to- the-previous (or next-to-the-next) selectable item, stays briefly, and continues moving 
back (or forward) through the list of selectable items for as long as the user continues holding the 
corresponding button 401 (or 403). This is simply an alternative to continuously rolling the roller 
control outlined above to quickly move the highlighting hack and forth through the list of selectable 
items. 

The device can also be designed such that the longer the button 401 or 403 is held, the faster the 
highlighting moves through the list of selectable items, up to some maximum speed (and with the 
speed accelerating in well defined increments from relatively slow to relatively fast). 

I turn now to a description of my invention embodied in a handset of the type described above. My 
invention can be embodied in a handset or other hand-held device with two or more displays, one of 



25 
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which can show content (such as an image, document, Web page, text, or other content) that includes 
one or more references to other content, such as referring to an image, document, Web page, text or 
other content. When a user indicates the desire to view referred-to content, for example, by using a 
button or control to interact with the first display, such as the direct-view display, and to click on, 
5 select, or open the referred-to content, then that referred-to content is displayed on one of the other 
displays, such as the microdisplay, for high resolution PC display-like viewing. So a user can use the 
direct-view display of the device to read an email or instant message that has an attached photo or a 
reference to a Web page URL and when the user clicks on or selects that attachment or Web page 
URL, the handset displays the attachment or Web page URL in the microdisplay. In addition, the 

10 device could show a full Web page (or a large portion of a Web page) in a microdisplay, and then 
show a subset of that Web page in a direct-view display. When the microdisplay is displaying a 
document or image (such as a Web page or application form) with items that can be selected or typed 
in, then at any given time, zero or one of those items is In focus" and that item is visually highlighted 
in some way, for example, by outlining that item in light gray. "In focus" means that if the user were to 

1 5 use a "select" control, by whatever name, on the handset or were to begin typing, then that "in focus" 
item would be the item selected, or, if it were a text entry box, it would be the item into which the 
characters would be typed. This is similar to the way items can be in "focus" in Web browsers or 
other applications on a desktop Windows PC. The item in "focus" is usually highlighted graphically in 
a desktop application, for example with a light gray outline. So if the microdisplay is displaying a 

20 document or image (such as a Web page or application form) and if one of the items on the document 
or image is "in focus." then a subset of the document or image surrounding the "in focus" item can be 
displayed on the direct-view display. This can provide a close-up view of the item that would be typed 
in or selected if the user were to type or use the handset's selection control. The user could see this 
area while holding the device a foot or two away, since it is displayed in the direct-view display. And 

25 this in turn would allow the user to see the buttons and controls on the device more easily as 

compared to holding the handset near-to-eye. This would allow the user to more easily type or use 
the controls on the phone while viewing that subset of the overall web-page in the direct-view display. 
This is best seen in Fig. 5. In that Figure the microdisplay 500 displays the web page while the subset 
thereof is displayed on the direct-view display 501. 

30 One way to implement this is to provide that the subset shown on the direct-view display would 

include both the "in focus" item and a text or bit-map representation of any text or graphic labels or 
symbols corresponding to that item on the form. For example, if the "in focus" item is a "Last name:" 
text-entry box on a Web-page form, then the direct-view display would show the label "Last name:" as 
well as the corresponding text-entry box, into which the user will type his or her last name. In 

35 practice, the item "in focus" (and hence the image subset shown on the direct-view display) can be 
changed by the user typing, depressing buttons, or using other controls on the device. For example, 
the user can use the roller control 102 in Fig. 1, or down/up buttons on Fig. 4, on the phone handset 
to move the focus back and forth through the list of items that can be selected or typed into on the 
current Web page or document. As the user uses the controls to change the focus, the direct-view 
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display should change to reflect the new focus, by showing the "region of interest" subset of the Web 
page or document surrounding that new in-focus item (or surrounding as much of the focus fits in the 
region of interest). This makes it easy for the user to always see the item that would be selected or 
typed-in, at any given moment, if the user were to select an item or type. Fig. 5 illustrates this concept. 
5 Fig. 5 is explained below. Additionally, the item in focus (and hence the image subset shown on the 
direct-view display) can be changed by the user speaking into the device, with the device itself, or a 
service accessed by the device over a network, using speech recognition software to interpret the 
user's spoken commands. Further still, the item in focus (and hence the image subset shown on the 
direct-view display) can be changed by software running on the computer processor within the device. 
10 This could be used to generate animations by having a large image made up of many smaller frames, 
and rapidly changing which subset of the larger image is displayed in the direct-view display. 

The item in focus can also be changed by the user typing, pressing buttons, speaking, or using other 
controls on the device and, as the focus changes, the image shown in the microdisplay auto-scrolls to 
keep the in-focus item in view. This is similar to what happens if a user views a large Web-page form 
on a computer monitor and the form extends past the bottom of the window in which the form is being 
displayed. That is, if the user uses the Tab button repeatedly to move the focus toward the end of the 
form, then when the focus goes past the bottom of the window to an item that can be typed-in or 
selected that appears on the part of the form below the window, the Web browser will typically auto- 
scroll the form upward within the window so that the new in-focus item will be visible within the 
window. In my invention, when a user types into a text-entry field shown on one display, and that text- 
entry field is also shown on another display (sometimes called "echoing"), then new text that appears 
in the first display will also appear in the other display reflecting the fact that both displays are 
showing the same text-entry field. For example, if the user is wearing a wearable display attached to 
a hand-held device with a direct-view display, then if the user types text into a field that is being 
shown in both the direct-view display and in the wearable display, the new text typed will appear on 
both displays - so the user won't be confused as they shift their eyes back and forth between the 
direct-view display and the wearable display's microdisplays. 

For example, if a user changes the state of a check-box, radio-button, or other "state" control that 
appears in more than one display, then all displays showing that same state control should show the 
30 state control in its current state. If a user types into a text-entry field shown on one or more displays, 
or uses arrow keys 1 13 of Fig. 1 or other controls to change the text-entry point or selection area, then 
if the text-entry point (or the end of the selection area) moves out of the subset of the document image 
visible in any given display, that display will automatically scroll the image so that the text-entry point 
(or the end of the selection area) will reappear in that display. This is similar to how documents 
35 automatically scroll in windows on desktop displays, except since this device may display different 
subsets of the same document in different displays, it must auto-scroll the image in different displays 
at different times. For example, if a relatively low-resolution direct-view display shows a small subset 
of a document shown in the much higher-resolution microdisplay. then the direct-view display may 
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have to auto-scroll several times as the user types a long message into that document, while the 
microdisplay could auto-scroll much less often. 

In contexts where the microdisplay is displaying a relatively high-resolution large apparent image (i.e. 
a virtual image, as discussed earlier), if the device uses a relatively low-resolution bit-map direct-view 
5 display, then at any given time the direct-view display can show a subset of the larger, higher- 
resolution apparent image shown in the microdisplay. That is, the direct-view display acts as a 
window into the larger image, showing a "region of interest" from the larger apparent image. 

Further, controls on the hand-held device, such as arrow buttons or a thumb roller control or similar 
controls, can move the region of interest window (i. e. the borders of the subset of the Web Page, 
10 application document, or other image shown in the direct-view display) down/up right/left or in other 
directions across the larger image. For example, at startup, the direct-view display might show an 
upper-left comer subset of the image seen in the microdisplay, which itself might be a subset of a 
larger scrollable document or Web page. Then the user can use the controls to have the direct-view 
show a different subset of the image seen in the microdisplay. 

1 5 Further, as the user types into a document or Web Page visible in the microdisplay, the subset of the 
document or Web Page that includes the text-entry point can automatically be scrolled into view in 
both the direct-view display and the microdisplay. That is, the "region of interest" can be automatically 
moved to surround the text-entry point Usually, the region of interest shown in the direct-view display 
is an area of bits around the current "focus" shown in the larger image displayed on the microdisplay. 

20 For example, a Web page has a set of selectable items such as links, buttons, editable text fields, and 
the like. As seen on desktop PCs, at any given time, one of the selectable items can be in "focus", and 
users can move the focus back and forth through the set of selectable items using the Tab Backward 
and Tab Forward key. (On desktop PC keyboards these are usually the same key: Tab-Backward = 
shift Tab Forward). Users can select that selectable item that is currently in focus by depressing a 

25 button (usually the space bar). As discussed in patent application S/N , a 

manufacturer can design a cell phone handset or other hand-held device on which a user could use a 
thumb-roller or other control to move the focus back and forth through the list of selectable items on 
the Web page or screen. As the user does this, if the Web Page or application document being 
viewed is larger than the image area visible in the microdisplay, then the image of the Web page or 

30 application document shown in the microdisplay should auto-scroll to always keep the focus in view in 
the microdisplay, just as occurs on desktop PCs when user tabs through selectable items (using the 
Tab key) on a large Web page displayed in a smaller Web browser window. Similarly, as the focus is 
moved, if the Web Page or application document being viewed is larger than the image area visible in 
the direct-view display, then the image of the Web page or application document shown in the direct- 

35 view display should auto-scroll to always keep the focus in view in the direct-view display. In a 

preferred embodiment, in accordance with the present invention, each display (when viewed) would 
show both the focus item (i. e. the selectable item that would get selected if the user hits the select 
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key) and the area immediately surrounding the focus item (which will often include text or image 
labels corresponding to the selectable item in focus, such as "Last Name:* labels). Each display thus 
acts as a window onto the Web Page or Document being viewed, with the direct-view display being a 
smaller window than the microdisplay. 

5 In contexts where the microdisplay is displaying a relatively high-resolution large apparent image of a 
Web Page, if the device uses a text-only (i. e., non-bit-map) direct-view display, then at any given time 
the direct-view display can show text corresponding to the item in focus on the Web Page. For 
example, if the item in focus is a text-entry field in a form on a Web page in which a user is supposed 
to type their last name, then the direct-view display can show any text immediately to the left (and/or 

1 0 right) of that text entry field (e. g. "Last Name: "). This text can often be extracted from the 

HTML that defines the Web Page {by interpreting the HTML to determine what text lies to the left of 
the selectable item in focus). But often it will not be possible to extract appropriate text corresponding 
to the Focus item (e. g. because no text corresponding to the focus item exists). For this reason, a 
preferred embodiment of this device would use a bit-map direct-view display, as discussed above, 

1 5 instead of a text-only direct-view display. 

As noted earlier, an important way this invention allows users to comfortably interact with full-size 
Web pages and other high-resolution or large content on pocket-size devices is by intelligently 
coordinating the content of direct-view displays with the content of microdisplays. Here are 
descriptions of a couple important embodiments of this invention. 



20 



25 



Referring to Fig. 5. when a user is viewing a Web page 502 on a device's microdisplay 500 held near- 
to-eye, the device should allow the user to position a cursor (such as the arrow cursor commonly seen 
when moving the mouse on a desktop computer) or a rectangular outline 503 or both (or some other 
indication of a "region of interest") over any chosen part of the Web page 502. Then when the user 
moves the device out to arms'-length the user should be able to view that "region of interest" on a 
direct-view display 501 on the device - i.e. viewing a subset of the larger image that appeared on the 
microdisplay. 



30 



In Fig. 5, the lines extending from the left comers of the Web page window 502 to the phone 
handset's embedded microdisplay 500 are meant to show that, when the microdisplay is held near the 
user's eye. the user could see a relatively large, high-resolution image like the Web page window 502. 
And the longer narrower lines extending from the "region of interest" rectangle 503 to the handset's 
direct-view display 501 are meant to show that a small portion of the Web page could be made visible 
on the direct-view display. In this example, the current region of interest is a small form on the Web 
page where the user could enter an address to look up a map. 



When the direct-view display 501 displays this region of interest (the area within the rectangle 503), 
35 the direct-view display acts as a window onto the larger image that would be seen on the microdisplay 
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if the microdispiay were he!d near-to-eye. While viewing the direct-view display 501 at arms -length 
(that is, at a normal reading distance), a user should be able to read text or interact with content in 
that region of interest, or use controls on the device to scroll the region of interest left, right, up, down, 
or other directions, to see different parts of the image other than the subset of the image currently 
5 shown in the direct-view display. If the user moves the region of interest while viewing the image in 
the direct-view display, then when the user returns to viewing the image in the microdispiay, the 
device can graphically indicate where the region of interest has moved - for example, by overlaying a 
rectangle 503 in the shape and size of the region of interest on top of the image seen in the 
microdispiay. The shape and size of the region of interest should generally be the same as, or close 
1 0 to, the pixel width and height of the direct- view display 501 . 

As illustrated in Fig. 5, the direct-view display 501 will typically have substantially fewer pixels than the 
microdispiay. For example, today's smartphones typically have displays that are less than 200 pixels 
wide and less than 250 pixels tall, and today's PDAs typically have displays that are less than 300 
pixels wide and less than 400 pixels tall. By contrast, Inviso Inc.'s current microdisplays are capable of 

1 5 displaying 800x600 pixel areas. The size and shape of the direct-view display can be used to define 
the size and shape of the region of interest. As the user uses controls on the device (or speech 
recognition, or other means) to move the region of interest around while viewing content on the 
microdispiay, the device can show the location of the region of interest by displaying a rectangle 503 
of that size on the image being viewed in the microdispiay 500. As the user moves the region of 

20 interest, the rectangle moves around the image. If the user then views the region of interest in the 
direct-view display 501 at arms'-length, and then uses controls on the device (or other means) to 
scroll the region of interest to other parts of the content being viewed (as described earlier), then 
when the user brings the device near-to-eye to view the content in the microdispiay, the rectangle 
showing the region of interest should be shown on the microdispiay in its new location on the image 

25 being viewed. 

Note that using a rectangle to show the region of interest is just one option. The device could use a 
range of techniques to indicate the region of interest on the microdispiay. Or device makers can 
choose not to explicitly mark the region of interest on the microdispiay image - choosing instead just 
to treat the area surrounding the cursor or surrounding highlighted M in focus" areas (discussed 
30 elsewhere in this patent) as the region of interest. Alternatively, if the region of interest is marked on 
the microdispiay (with a rectangle, for example), then device makers can give users the option of 
hiding that mark or showing that mark. 

As discussed above, it is sometimes useful to display an icon or other visual cue on a direct-view 
display when there is new content for the user to look at in the microdispiay. and vice-versa, since it 
35 can be hard to see content on the microdispiay when looking at the direct-view display, and vice- 
versa. Fig. 6 illustrates a preferred embodiment for providing this visual cue. The sample phone on 
the right side of Fig. 6 has a microdispiay 604 near the top and a direct-view display 600 below the 
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microdisplay. The direct-view display 601 is shown in close-up on the left side of Fig. 6. In this 
example, the user (Matt) has received an instant message. The user would typically read instant 
messages at arms '-length (i.e. normal reading distance) on a direct-view display 600 (shown in close 
up on the left 601). The instant message in this example includes a URL 603 referring to a Web page 
5 with a map (similar to the Web page shown in Fig. 5.) In this preferred embodiment, when the user 
operates controls on the phone to open the Web page associated with the URL, the device will display 
a flashing eye icon 602 on the direct-view display while the device is downloading and preparing that 
Web page for display. And when the Web page is ready for viewing on the microdisplay 604, the 
device would display a steady (i.e. non-flashing) eye icon 602. The form of the indication is not 

10 important - it could be a different icon, or text, or some other visual cue. and it could flash or not-flash. 
The important point is that, when a user is holding a device at arms-length, there are times when it is 
difficult for them to know when there have been significant changes to the content available for 
viewing on the near-to-eye microdisplay (since it can be difficult to see content on a microdisplay at 
arms'-length), so at those times, it is useful to provide visual cues on the direct-view display. Similarly. 

15 if the user is viewing content on the microdisplay. in some cases it could be useful to provide visual 

cues on the microdisplay when there is significant new content available for viewing on the direct-view 
display, since the user may not think to look at the direct-view display while looking at the 
microdisplay (even if the microdisplay is on a head-mounted wearable display rather than embedded 
in the hand-held device with the direct-view display). 

20 In the foregoing discussion, we have used Web pages as our primary example of content that can be 
accessed more easily by intelligently coordinating the images displayed on a device's direct-view 
display and microdisplay. But it will be appreciated by those skilled in the art that the same techniques 
can be applied to a wide range of content, including computer applications, documents (including 
spreadsheets, presentations, and word-processor documents, among others), photos, video, and 

25 more. 

The present invention can be used when designing devices with a wide range of form-factors and 
configurations. For example, a device with a direct-view display on its face and a microdisplay 
embedded near the top of the device works well. But device makers could choose to embed or attach 
one or more microdisplays to any part of the device (e.g. on a hinge on the bottom of the phone or on 
30 the side) and still use this invention to intelligently coordinate content on the device's multiple displays 
and thereby improve the ergonomics of the device. 



While the foregoing has been with reference to particular embodiments of the invention, it will be 
appreciated by those skilled in the art that changes in these embodiments may be made without 
departing from the principles and spirit of the invention, the scope of which is defined by the appended 
35 claims. 
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I CLAIM: 

1 . A phone handset or other hand-held device which displays images, said device having a face, 
said face having two or more displays for displaying said images and control elements for 
controlling said displays, wherein a user can operate said control elements to affect visual 

5 changes on one or more of the displays. 

2. The devices of claim 1 in which at least one display is a direct-view display. 

3. The device of claim 1 in which at least one of the displays is a microdisplay. 

4. The device of claim 1 in which the device has exactly two displays. 

5. The device of claim 4 in which the device includes one direct-view display and one 
10 microdisplay. 

6. The device of claims 1 or 2 or 3 or 4 or 5 that operates on functions transmitted remotely from 
said device. 

7. The device of claim 6 wherein such functions are selected from the group consisting of instant 
messaging, email, contact management, schedule management and phone calling. 

15 8. The device of claim 6 wherein such at least one of said functions is displayed in the direct- 
view display. 

9. The device of claim 8 wherein the displayed function includes an attachment that can be 
displayed in the microdisplay. 

10. The device of claim 9 further including a distance sensor wherein the user affects displaying 
20 said attachment in the microdisplay by bringing the microdisplay to the user's eye. 

1 1 . The device of claim 1 in which, when an image is displayed in one display but not the others, 
then the device can display a text or graphics indication on one or more of the displays that is 
not displaying that image, said indication indicating that one of the other displays is displaying 
that image. 

25 12. The device of claim 1 1 in which at least one display is a direct-view display. 

1 3. The device of claim 1 1 in which at least one of the displays is a microdisplay. 

14. The device of claim 1 1 in which the device has exactly two displays. 

15. The device of claim 14 in which the device includes one direct-view display and one 
microdisplay. 
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16. The device of claims 9 or 10 in which (1) the displayed function is taken from the group of 
functions consisting of instant messaging, email, contact management, schedule 
maintenance, calculation functions and placing telephone calls, viewing Internet content, and 
viewing other types of documents, (2) the attachment is an attachment taken from the group 

5 of attachments consisting of a Web page URL, a photo image, a spreadsheet, a presentation, 

an Adobe Acrobat file, a Word Processor document, and other types of content, and (3) the 
controls are capable of operation by a user to display that attachment in the microdisplay. 

17. The device of claim 16 further including control elements which, when activated, cause the 
device to display an indication in the direct-view display that indicates to the user when all or 

10 part of said attachment is being downloaded or prepared for display on the microdisplay, or 

cause the device to display an indication in the direct-view display that indicates when the 
attachment is ready for viewing on the microdisplay. 

18. A phone handset or other hand-held device with two or more displays, said device having 
controls operable by a user to (1) cause one of the displays to display first content that 

15 includes one or more references to other content, and (2) cause that referred-to content to be 

displayed in one of the other displays. 

19. The device of claim 18 wherein said first content is content selected from the group consisting 
of an image, a document, a Web page or text and the referred content is content selected 
from the group consisting of an image, a document, a Web page, text or other content. 

20 20. The device of claim 18 in which at least one display is a direct-view display. 

21 . The device of claim 18 in which at least one of the displays is a microdisplay. 

22. The device of claim 18 in which the device has exactly two displays. 

23. The device of claim 22 in which the device includes one direct-view display and one 
microdisplay. 

25 24. A phone handset or other hand-held device which displays images, said device having two or 

more displays for displaying said images and control elements for controlling said displays, 
wherein a user can operate said control elements to cause a first display to display an image 
and cause a second display to display a region of interest subset of said image. 

25. The device of claim 24 wherein the device can display substantially a full Web page in one 
3 0 display and display a close up view of one subset of that Web page in another of the displays. 

26. The device of claim 24 in which at least one display is a direct-view display. 

27. The device of claim 24 in which at least one of the displays is a microdisplay. 
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28. The device of claim 24 in which the device has exactly two displays. 

29. The device of claim 24 in which the device includes one direct-view display and one 
microdisplay. 

30. The device of claim 23 in which, when a first display is displaying first content having items 
5 that can be selected or typed in, one of such items is in-focus. 

31 . The device of claim 30 wherein said one in-focus item is highlighted by outlining that item in a 
visually recognizable shade. 

32. The device of claim 30 having a select function whereby a user may select said one item. 

33. The device of claim 32 further including one or more face-keys which, when depressed by a 
10 user, causes a character to be typed into said one item if said one item is an editable text 

field. 

34. The device of claim 30 further including controls operable by a user that cause, if the device 
has prepared an image having one item in-focus for display in the microdisplay, a subset of 
said image surrounding the in-focus item to be displayed in the direct-view display. 

15 35. The device of claim 34 in which the subset displayed in the direct-view display is a close up 

view of said one item. 

36. The device of claim 34 in which the subset displayed in the direct-view display includes both 
the in-focus item and a text or bit-map representation of text or graphic labels or symbols 
corresponding to that item. 

20 37. The device of claim 36 in which the in-focus item is a "Last Name" text entry box on a Web- 
page form and the subset includes the label "Last Name" and the corresponding text-entry 
box into which the last name will be typed. 

38. The device of claim 34 including controls operable by a user to cause the subset displayed in 
the direct-view display to be changed by user operation. 

25 39. The device of claim 36 in which the device includes one control element from the group 

consisting of a roller control or up/down buttons or multi-speed up/down controls capable of 
operation by the user to cause the in-focus item to change from the currently in-focus item to 
the previous or next item that can be selected or typed in. 

40. The device of claim 39 including software that causes the image displayed in the direct-view 
30 display to concurrently change when the in-focus item changes from one selectable item to a 

new in-focus item. 
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41. The device of claim 40 in which said image is changed by displaying a subset of the image 
surrounding all or part of said new in-focus item. 

42. The device of claim 34 including a microphone and voice recognition software that can allow 
a user to cause the in-focus item to change from one selectable item to another one by the 
user speaking into the device. 

43. The device of claim 34 including a computer processor in which both said item in-focus in said 
microdisplay and said image subset displayed in the direct-view display can be changed by 
software running on said computer processor within the device. 

44. The device of claim 43 wherein said change generates animations by having a large image 
made up of many smaller frames, and rapidly changing which subset of the larger image is 
displayed in the direct-view display. 

45. The device of claim 43 in which the item in-focus can be changed by the user typing, 
depressing buttons, speaking, or using other controls on the device, and in which, as the 
focus changes, if the microdisplay is displaying an image, the image shown in the 
microdisplay auto-scrolls to keep the in-focus item in view, and, if the direct-view display is 
displaying an image, the image shown in the direct-view display auto-scrolls to keep the in- 
focus item in view. 

46. The device of claim 24 having a plurality of face-keys capable of being depressed by a user to 
generate new text into a text-entry field displayed in a first display, which text-entry field is 
also shown on a second display such that the new text that appears in said first display will 
also appear in said second display. 

47. The device of claim 24 including at least one control button capable of being depressed by a 
user to generate a changed state of a check box, radio-button, or other state control that 
appears in more than one display, and concurrently changing to said changed state all 
displays showing that same check box, radio-button or other state control. 

48. The device of claim 24 capable of displaying a subset of said image in at least one of said 
displays and further having at least one of the group consisting of (1) a plurality of face-keys 
capable of being depressed by a user to generate text typed into a text-entry field shown on 
said one or more displays, and (2) arrow keys or other controls which, when actuated by a 
user, change said text-entry point, wherein if the text-entry point moves out of the subset of 
said image visible in any given display, then that display will automatically scroll the image so 
that the text-entry point will reappear in that display. 
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49. The device of claim 29 wherein the microdisplay displays a relatively high-resolution large 
apparent image, the direct-view display is a low-resolution bit-map direct-view display, said 
direct view display acting as a window into said larger apparent image. 

50. The device of claim 29 wherein the first display is a microdisplay and the second display is a 
5 direct-view display, said device further including controls operable by a user to move the 

borders of said region of interest subset displayed in the direct-view display down or up or 
right or left or other directions across the image displayed in the microdisplay. 

51 . The device of claim 50 further including face-keys for entering text, the image is a document 
or Web Page and, as the user operates said face-keys to enter text into said document or 

10 Web page, the subset of the document or Web Page that includes the text-entry point is 

automatically scrolled into view in both the direct view display and the microdisplay. 

52. The device of claim 51 in which the subset displayed in the direct-view display is an area of 
bits containing ail or part of the current in-focus area of the image displayed in the 
microdisplay. 

1 5 53. A computer implemented process generating codes for interpreting the operation of control 

elements on a phone handset or other hand-held device which displays images, said phone 
handset or other device having (1)a face, said face having two or more displays for 
displaying said images and (2) control elements operable by a user for controlling said 
displays, said codes effecting visual changes on one or more of the displays in response to 

20 such operation. 

54. The process of claim 53 in which at least one display is a direct-view display. 

55. The process of claim 54 in which at least one of the displays is a microdisplay 

56. The process of claim 55 in which the device has exactly two displays. 

57. The process of claim 56 in which the device includes one direct-view display and one 
25 microdisplay. 

58. The process of claims 53 or 54 or 55 or 56 or 57 that operates on functions transmitted 
remotely from said device. 

59. The process of claim 58 wherein such functions are selected from the group consisting of 
instant messaging, email, contact management, schedule management and phone calling. 

30 60. The process of claim 58 wherein such at least one of said functions is displayed in the direct- 
view display. 
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61. The process of claim 60 wherein the displayed function includes an attachment that can be 
displayed in the microdisplay. 

62. The process of claim 61 wherein said control elements include a distance sensor, said codes 
effecting displaying said attachment in the microdisplay in response to the user bringing said 
distance sensor to the user's eye. 

63. The process of claim 53 in which, in response to an image being displayed in one display but 
not at least one of the others, the codes effect displaying a text or graphics indication on one 
or more of the displays that is not displaying that image, said indication indicating that one of 
the other displays is displaying that image. 

64. The process of claim 61 or 62 in which (1)the function is taken from the group of functions 
consisting of instant messaging, email, contact management, schedule maintenance, 
calculation functions, placing telephone calls, viewing Internet content, and viewing other 
types of documents, (2) the attachment is an attachment taken from the group of attachments 
consisting of a Web page URL or a photo image, a spreadsheet, a presentation, an Adobe 
Acrobat file, a Word Processor document, and other types of content, and (3) the control 
elements include first controls capable of operation by a user to display that attachment in the 
microdisplay, said codes effecting such display of such attachment in response to operation 
of said second controls. 

65. The process of claim 64 wherein said control elements includes second control elements 
which, when activated by the user, display an indication in the direct-view display that 
indicates to the user that the attachment has been opened and is being displayed in the 
microdisplay and said codes effecting such display of such indication in response to said 
activation. 

66. A computer implemented process generating codes for interpreting the operation of control 
elements on a phone handset or other hand-held device having two or more displays, said 
phone handset or other device having first control elements operable by a user to (1) cause a 
first display to display first content that includes one or more references to other content, and 
(2) to cause that referred-to content to be displayed in a second display, and said codes 
effecting each said displays in response to operation of said controls. 

67. The process of claim 66 wherein said first content is content selected from the group 
consisting of an image, a document, a Web page, text or other content, and the referred-to 
content is content selected from the group consisting of an image, a document, a Web page, 
text or other content 



68. 



The process of claim 67 in which the device includes one direct-view display and one 
microdisplay. 



WO 02/079913 



-24- 



PCT/US02/10458 



69. A computer implemented process generating codes for interpreting the operation of control 
elements on a phone handset or other hand-held device which displays images, said phone 
handset or other device having (1)two or more displays for displaying said images and 
(2) control elements for controlling said displays, wherein a user can operate said control 

5 elements to cause a first display to display an image and cause a second display to display a 

region of interest subset of said image, and said codes effect displaying of said subset in 
response to the operation of said control elements. 

70. The process of claim 69 wherein said control elements include first controls operable by a 
user to (1) display substantially a full Web page in one display and (2) display a close up view 

1 0 of one subset of that Web page in another of the displays, said codes effecting said displaying 

in said one display and said displaying in said another of the displays in response to the 
operation of said first controls. 

71. The process of claim 69 in which the device includes one direct-view display and one 
microdisplay. 

15 72. The process of claim 69 wherein said control elements include second controls operable by a 

user which, when a first display is displaying first content having items that can be selected or 
typed in, cause one of such items to be in-focus, said codes effecting said one of such items. 

73. The process of claim 72 wherein, in response to the user operating said second controls, said 
codes effect said one in-focus item being highlighted by outlining that item in a visually 

20 recognizable shade. 

74. The process of claim 73 wherein the device has a select function operable by a user to select 
said one in-focus item, said codes effecting selecting said one item into in response to the 
operation of said select function. 

75. The process of claim 74 wherein the device further includes one or more face-keys which, 
25 when depressed by a user, causes a character to be typed into said one item if said one item 

is an editable text field, said codes effecting said character being so typed in response to the 
user depressing said one or more face-keys. 

76. The process of claim 69 wherein said control elements include third controls operable by a 
user to cause, if the device has prepared an image having one item in-focus for display in the 

30 microdisplay,. a subset of said image surrounding the in-focus item to be displayed in the 

direct-view display, said codes effecting said subset of said image being so displayed in 
response to the operation of said third controls. 



77. 



The process of claim 76 in which the subset displayed in the direct-view display is a close up 
view of said one item. 
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78. The process of claim 76 in which the subset displayed in the direct- view display includes both 
the in-focus item and a text or bit-map representation of text or graphic labels or symbols 
corresponding to that item. 

79. The process of claim 76 in which the in-focus item is a "Last Name" text entry box on a Web- 
page form and the subset includes the label "Last Name" and the corresponding text-entry 
box into which the last name will be typed. 

80. The process of claim 76 wherein the control elements include fourth controls operable by a 
user to cause the subset displayed in the direct-view display to be changed by user operation, 
said codes effecting said change in response to said user operation . 

81. The process of claim 78 in which the device includes at least one control element from the 
group consisting of a roller control or up/down buttons or multi-speed up/down controls 
capable of operation by the user to cause the in-focus item to change from the current in- 
focus item to the previous or next item that can be selected or typed in. 

82. The process of claim 81 wherein the process includes software that causes the image 
displayed in the direct-view display to concurrently change when the in-focus item changes 
from one selectable item to a new in-focus item. 

83. The process of claim 82 in which said image is changed by displaying a subset of the image 
surrounding all or part of said new in-focus item. 

84. The process of claim 62 in which said phone handset or other hand-held device further 
includes a menu button and, if a user depresses said menu button while said distance sensor 
senses that the microdisplay on said device is close to an object, then the device will display 
a menu on the microdisplay, and otherwise the device will display said menu on the direct- 
view display. 

85. The process of claim 76 wherein the device includes a microphone and voice recognition 
software that allows a user to cause the in-focus item to be changed from one selectable item 
to another one by the user speaking into the device, said codes effecting said change in 
response to the user so speaking. 

86. The process of claim 76 wherein the device includes a computer processor in which both said 
item in-focus in said microdisplay and said image subset displayed in the direct-view display 
can be changed by software running on said computer processor within the device. 

87. The process of claim 85 wherein said change generates animations by having a large image 
made up of many smaller frames, and rapidly changing which subset of the larger image is 
displayed in the direct-view display. 
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88. The process of claim 69 in which the item in-focus can be changed by the user typing, 
depressing buttons, speaking, or using other controls on the device, and in which, as the 
focus changes, if the microdisplay is displaying an image, the image shown in the 
microdisplay auto-scrolls to keep the in-focus item in view, and, if the direct-view display is 

5 displaying an image, the image shown in the direct-view display auto-scrolls to keep the in- 

focus item in view. 

89. The process of claim 69 wherein the device has a plurality of face-keys capable of being 
depressed by a user to generate new text into a text-entry field displayed in a first display, 
which text-entry field is also shown in a second display such that the new text that appears in 

10 said first display will also appear in said second display, said codes effecting said new text 

appearing in said first display an in said second display in response to the user so depressing 
said face keys. 

90. The process of claim 69 wherein the device includes at least one control button capable of 
being depressed by a user to generate a changed state of a check box, radio-button, or other 

15 state control that appears in more than one display, and concurrently changing to said 

changed state all displays showing that same check box, radio-button or other state control. 

91 . The process of claim 69 in which the device is capable of displaying a subset of said image in 
at least one of said displays and further having at least one of the group consisting of (1) a 
plurality of face-keys capable of being depressed by a user to generate text typed into a text- 

20 entry field shown on said one or more displays, and (2) arrow keys or other controls which, 

when actuated by a user, change said text-entry point, wherein if the text-entry point moves 
out of the subset of said image visible in any given display, then said codes effect said any 
given display automatically scrolling the image so that the text-entry point will reappear in said 
any given display. 

25 92. The process of claim 71 wherein the microdisplay displays a relatively high-resolution large 

apparent image, the direct-view display is a low-resolution bit-map direct-view display, said 
direct-view display acting as a window into said larger apparent image. 

93. The process of claim 69 wherein a first display is a microdisplay and a second display is a 
direct-view display, and a subset of said control elements are operable by a user to move the 

30 borders of said region of interest subset displayed in the direct-view display down or up or 

right or left or other directions across the image displayed in the microdisplay, said codes 
effecting said movement of said borders in response to the operation of said subset of said 
control elements. 

94. The process of claim 92 wherein the device further includes face-keys operable by a user to 
35 enter text, the image is a document or Web Page having a text entry point, said codes 

effecting the subset of the document or Web Page that includes the text-entry point being 
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automatically scrolled into view in both the direct-view display and the microdisplay in 
response to the user operating said face-keys to enter text into said document or Web page. 

95. The process of claim 92 in which the subset displayed in the direct-view display is an area of 
bits containing all or part of the current in-focus area of the image displayed in the 
microdisplay. 

96. The process of claim 63 wherein the device includes one direct view display and one 
microdisplay. 

97. A storage medium having embedded therein a computer implemented process generating 
codes for interpreting the operation of control elements on a phone handset or other hand- 
held device which displays images, said phone handset or other device having (1)a face, 
said face having two or more displays for displaying said images and (2) control elements 
operable by a user for controlling said displays, said code effecting visual changes on one or 
more of the displays in response to such operation. 

98. The storage medium of claim 97 in which at least one display is a direct-view display. 
1 5 99. The storage medium of claim 98 in which at least one of the displays is a microdisplay 

100. The storage medium of claim 99 in which the device has exactly two displays; 

101. The storage medium of claim 100 in which the device includes one direct-view display and 
one microdisplay. 

102. The storage medium of claims 97 or 98 or 99 or 100 or 101 that operates on functions 
20 transmitted remotely from said device. 

103. The storage medium of claim 102 wherein such functions are selected from the group 
consisting of instant messaging, email, contact management, schedule management and 
phone calling. 

104. The storage medium of claim 102 wherein such at least one of said functions is displayed in 
25 the direct-view display. 

105. The storage medium of claim 104 wherein the displayed function includes an attachment that 
can be displayed in the microdisplay. 

106. The storage medium of claim 105 wherein said control elements include a distance sensor, 
said codes effecting displaying said attachment in the microdisplay in response to the user 

30 bringing said distance sensor to the user's eye. 
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107. The storage medium of claim 97 in which, in response to an image being displayed in one 
display but not at least one of the others, the codes effect displaying a text or graphics 
indication on one or more of the displays that is not displaying that image, said indication 
indicating that one of the other displays is displaying that image. 

108. The storage medium of claim 105 or 106 in which (1) the function is taken from the group of 
functions consisting of instant messaging, email, contact management, schedule 
maintenance, calculation functions, placing telephone calls, viewing Internet content, and 
viewing other types of documents. (2) the attachment is an attachment taken from the group 
of attachments consisting of a Web page URL or a photo image, a spreadsheet, a 
presentation, an Adobe Acrobat file, a Word Processor document, and other types of content, 
and (3) the control elements include first controls capable of operation by a user to display 
that attachment in the microdisplay, said codes effecting such display of such attachment in 
response to operation of said second controls. 

109. The storage medium of claim 108 wherein said control elements includes second control 
elements which, when activated by the user, display an indication in the direct-view display 
that indicates to the user that the attachment has been opened and is being displayed in the 
microdisplay and said codes effecting such display of such indication in response to said 
activation. 

110. A storage medium having embedded therein a computer implemented process generating 
codes for interpreting the operation of control elements on a phone handset or other hand- 
held device having two or more displays, said phone handset or other device having first 
control elements operable by a user to (1) cause a first display to display first content that 
includes one or more references to other content, and (2) to cause that referred-to content to 
be displayed in a second display, and said codes effecting each said displays in response to 
operation of said controls. 

111. The storage medium of claim 110 wherein said first content is content selected from the group 
consisting of an image, a document, a Web page or text and the referred-to content is content 
selected from the group consisting of an image, a document, a Web page, text or other 
content. 

112. The storage medium of claim 111 in which the device includes one direct-view display and 
one microdisplay. 

113. A storage medium having embedded therein a computer implemented process generating 
codes for interpreting the operation of control elements on a phone handset or other hand- 
held device which displays images, said phone handset or other device having (1)two or 
more displays for displaying said images and (2) control elements for controlling said 
displays, wherein a user can operate said control elements to cause a first display to display 



WO 02/079913 



-29- 



PCT7US02/10458 



an image and cause a second display to display a region of interest subset of said image, and 
said codes effect displaying of said subset in response to the operation of said control 
elements. 

114. The storage medium of claim 113 wherein said control elements include first controls 
operable by a user to (1) display substantially a full Web page in one display and (2) display a 
close up view of one subset of that Web page in another of the displays, said codes effecting 
said displaying in said one display and said displaying in said another of the displays in 
response to the operation of said first controls. 

115. The storage medium of claim 113 in which the device includes one direct-view display and 
one microdisplay. 

116. The storage medium of claim 113 wherein said control elements include second controls 
operable by a user which, when a first display is displaying first content having items that can 
be selected or typed in, cause one of such items to be in-focus, said codes effecting said one 
of such items. 

117. The storage medium of claim 116 wherein, in response to the user operating said second 
controls, said codes effect said one in-focus item being highlighted by outlining that item in a 
visually recognizable shade. 

118. The storage medium of claim 1 17 wherein the device has a select function operable by a user 
to select said one in-focus item, said codes effecting selecting said one item in response to 
the operation of said select function. 

119. The storage medium of claim 118 wherein the device further includes one or more face-keys 
which, when depressed by a user, causes a character to be typed into said one item if said 
one item is an editable text field, said codes effecting said character being so typed in 
response to the user depressing said one or more face-keys. 

120. The storage medium of claim 113 wherein said control elements include third controls 
operable by a user to cause, if the device has prepared an image having one item in-focus for 
display in the microdisplay, a subset of said image surrounding the in-focus item to be 
displayed in the direct-view display, said codes effecting said subset of said image being so 
displayed in response to the operation of said third controls. 

121 . The storage medium of claim 120 in which the subset displayed in the direct-view display is a 
close up view of said one item. 

122. The storage medium of claim 120 in which the subset displayed in the direct-view display 
includes both the in-focus item and a text or bit-map representation of text or graphic labels or 
symbols corresponding to that item. 
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123. The storage medium of claim 120 in which the in-focus item is a "Last Name" text entry box 
on a Web-page form and the subset includes the label "Last Name" and the corresponding 
text-entry box into which the last name will be typed. 

124. The storage medium of claim 120 wherein the control elements include fourth controls 
5 operable by a user to cause the subset displayed in the direct-view display to be changed by 

user operation, said codes effecting said change in response to said user operation. 

125. The storage medium of claim 122 in which the device includes at least one control element 
from the group consisting of a roller control or up/down buttons or multi-speed up/down 
controls capable of operation by the user to cause the in-focus item to change from the 

1 o current in-focus item to the previous or next item that can be selected or typed in. 

126. The storage medium of claim 125 wherein the process includes software that causes the 
image displayed in the direct-view display to concurrently change when the in-focus item 
changes from one selectable item to a new in-focus item. 

127. The storage medium of claim 126 in which said image is changed by displaying a subset of 
15 the image surrounding all or part of said new in-focus item. 

128. The storage medium of claim 120 wherein the device includes a microphone and voice 
recognition software that allows a user to cause the item in-focus to be changed from one 
selectable item to another one by the user speaking into the device, said codes effecting said 
change in response to the user so speaking. 

20 1 29. The storage medium of claim 1 20 wherein the device includes a computer processor in which 

both said item in-focus in said microdisplay and said image subset displayed in the direct- 
view display can be changed by software running on said computer processor within the 
device. 

130. The storage medium of claim 129 wherein said change generates animations by having a 
25 large image made up of many smaller frames, and rapidly changing which subset of the larger 

image is displayed in the direct-view display. 

131. The storage medium of claim 113 in which the item in-focus can be changed by the user 
typing, depressing buttons, speaking, or using other controls on the device, and in which, as 
the focus changes, if the microdisplay is displaying an image, the image shown in the 

30 microdisplay auto-scrolls to keep the in-focus item in view, and, if the direct-view display is 

displaying an image, the image shown in the direct-view display auto-scrolls to keep the in- 
focus item in view. 

132. The storage medium of claim 1 13 wherein the device has a plurality of face-keys capable of 
being depressed by a user to generate new text into a text-entry field displayed in a first 
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display, which text-entry field is also shown in a second display such that the new text that 
appears in said first display will also appear in said second display, said codes effecting said 
new text appearing in said first display an in said second display in response to the user so 
depressing said face keys. 

133. The storage medium of claim 113 wherein the device includes at least one control button 
capable of being depressed by a user to generate a changed state of a check box, radio- 
button, or other state control that appears in more than one display, and concurrently 
changing to said changed state all displays showing that same check box, radio-button or 
other state control. 

1 34. The storage medium of claim 1 1 3 in which the device is capable of displaying a subset of said 
image in at least one of said displays and further having at least one of the group consisting 
of (1) a plurality of face-keys capable of being depressed by a user to generate text typed into 
a text-entry field shown on said one or more displays, and (2) arrow keys or other controls 
which, when actuated by a user, change said text-entry point, wherein if the text-entry point 
moves out of the subset of said image visible in any given display, then said codes effect said 
any given display automatically scrolling the image so that the text-entry point will reappear in 
said any given display. 

135. The storage medium of claim 115 wherein the microdisplay displays a relatively high- 
resolution large apparent image, the direct-view display is a low-resolution bit-map direct-view 
display, said direct-view display acting as a window into said larger apparent image. 

136. The storage medium of claim 113 wherein a first display is a microdisplay and a second 
display is a direct-view display, and a subset of said control elements are operable by a user 
to move the borders of said region of interest subset displayed in the direct-view display down 
or up or right or left or other directions across the image displayed in the microdisplay. said 
codes effecting said movement of said borders in response to the operation of said subset of 
said control elements. 

137. The storage medium of claim 136 wherein the device further includes face-keys operable by a 
user to enter text, the image is a document or Web Page having a text entry point, said codes 
effecting the subset of the document or Web Page that includes the text-entry point being 
automatically scrolled into view in both the direct-view display and the microdisplay in 
response to the user operating said face-keys to enter text into said document or Web page. 

138. The storage medium of claim 136 in which the subset displayed in the direct-view display is 
an area of bits containing all or part of the current in-focus area of the image displayed in the 
microdisplay. 
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139. The storage medium of claim 107 wherein the device has one direct view display and one 
microdisplay. 

140. The device of claim 1 or claim 24, wherein said device is a phone, smartphone, PDA, hand- 
held computer, or watch. 

141. The device of claim 1 or claim 24, wherein the device can receive and transmit data 
wirelessly. 

142. The device of claim 1 or claim 24, wherein the device includes a microdisplay embedded in or 
attached near the top of the face of the device, a keypad or touchpad near the bottom of the 
face of the device, and a direct-view display visible on the face of the device between the 
microdisplay and the touchpad. 

143. The device of claim 1 or claim 24, wherein the device includes a first part including a direct- 
view display, a second part including a microdisplay and a keypad or touchpad, and a hinge 
connecting said first part to said second part, wherein said microdisplay is between said hinge 
and said keypad or touchpad. 



WO 02/079913 



1/6 



PCT/US02/10458 




CD r- OJ CO ^ LO CD 

CD CD CD CD CD CD CD CD 





.1 


— 4*— - % 1 










i J 













CD 



0207991 3 A2 I > 



SUBSTITUTE SHEET (RULE 26) 



WO 02/079913 



PCT/US02/10458 

2/6 





SUBSTITUTE SHEET (RULE 26) 



nnnmn - ** » «"» • 







CO 
CD 



SUBSTITUTE SHEET (RULE 26) 

SDOCID:<WO 0207991 3 A2 I > 



WO 02/079913 



4/6 



PCT7US02/10458 



400 




FIG. 4 

SUBSTITUTE SHEET (RULE 26) 



WO 02/079913 



PCT/US02/10458 




SUBSTITUTE SHEET (RULE 26) 



0207991 3 A2 I > 



1 * 

I 



WO 02/079913 PCT/US02/10458 

6/6 




SUBSTITUTE SHEET (RULE 26) 



(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 

International Bureau 

(43) International Publication Date 
10 October 2002 (10.10.2002) 




III 




111 




PCT 



(10) International Publication Number 

WO 02/079913 A3 



(51) International Patent Classification 7 : H04B 1/38, 

H04M 1/00 

(21) International Application Number: PCT/US02/10458 

(22) International Filing Date: 2 April 2002 (02.04.2002) 

(25) Filing Language: English 

(26) Publication Language: English 

(30) Priority Data: 

60/280,79 1 2 April 200 1 (02.04.200 1 ) US 

10/1 13,239 1 April 2002 (01.04.2002) US 

(71) Applicant and 

(72) Inventor: PALLAKOFF, Matthew, G. [US/US]; 456 
Mountain Laurel Court, Mountain View, CA 94043 (US). 



(74) Agent: LEAL, Peter, R.; Gray Cary Ware & Freidenrich 
LLP, Patent Dept., 1755 Embarcadero Road, Palo Alto, CA 
94303 (US). 

(81) Designated States (national): AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CO, CR, CU, 
CZ, DE, DK, DM, DZ, EC, EE, ES, FI, GB, GD, GE, GH, 
GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, 
LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, 
MX, MZ, NO, NZ, OM, PH, PL, PT, RO, RU, SD, SE, SG, 
SI, SK, SL, TJ, TM, TR, TT, TZ, UA, UG, UZ, VN, YU, 
ZA, ZM, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW), 
Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), 
European patent (AT, BE, CH, CY, DE, DK, ES, FT, FR, 
GB, GR, IE, IT, LU, MC, NL, PT, SE, TR), OAPI patent 
(BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, ML, MR, 
NE, SN, TD, TG). 

[Continued on next page] 



(54) Title: COORDINATING IMAGES DISPLAYED ON DEVICES WITH TWO OR MORE DISPLAYS 



114 




03 

04 

05 
OB 
07 



CD CD CD 
<SI>C£>CD 




117 



(57) Abstract: A hand held electronic device (Fig. IB) having at least 
two displays is disclosed. At least one display is direct-view display for 
viewing at normal reading distance of arms' -length viewing. The other 
display is a microdisplay. a tiny display with magnifying optical elements, 
for viewing larger, higher-resolution images when the microdisplay is po- 
sitioned close to the eye. The invention allows coordinating microdisplays 
and direct-view displays in ways that allow people to comfortably access 
and interact with full-page Web content on pocket-size devices. When a 
user is viewing a Web page on a device's microdisplay held near-to-eye, 
the device allows the user to position a cursor or a rectangular outline on a 
particular part of the Web page, and then when the user moves the device 
out to arms* -length viewing, the user should be able to view that region of 
interest on the direct-view display. 
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